Award  Number:  W81XWH-04-2-0050 


AD 


TITLE:  NMMC  Picture  Archiving  and  Communication  Systems  (PACS) 


PRINCIPAL  INVESTIGATOR:  Robert  C.  Becker,  M.D. 


CONTRACTING  ORGANIZATION:  North  Mississippi  Medical  Center  Inc. 

Tuppelo,  MS  38801-4996 


REPORT  DATE:  April  2007 

TYPE  OF  REPORT:  Annual 


PREPARED  FOR:  U.S.  Army  Medical  Research  and  Materiel  Command 
Fort  Detrick,  Maryland  21702-5012 


DISTRIBUTION  STATEMENT:  Approved  for  Public  Release; 

Distribution  Unlimited 


The  views,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and 
should  not  be  construed  as  an  official  Department  of  the  Army  position,  policy  or  decision 
unless  so  designated  by  other  documentation. 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing 
this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202- 
4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently 
valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1.  REPORT  DATE  (DD-MM-YYYY) 

01-04-2007 

2.  REPORT  TYPE 

Annual 

3.  DATES  COVERED  ( From  -  To) 

29  Mar  2006 -28  Mar  2007 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

NMMC  Picture  Archiving  and  Communication  Systems  (PACS) 

5b.  GRANT  NUMBER 

W81 XWH-04-2-0050 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

Robert  C.  Becker,  M.D. 

5e.  TASK  NUMBER 

E-Mail:  rbecker(5)nmhs.net 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

North  Mississippi  Medical  Center  Inc. 

Tuppelo,  MS  38801-4996 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Medical  Research  and  Materiel  Command 

Fort  Detrick,  Maryland  21702-5012 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  Public  Release;  Distribution  Unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSIKACI 

Most  deliverables  for  this  project  period  were  met:  'PACS  equipment  became  fully  operational  at  13  primary  care  clinics  in  rural  Mississippi  (2  clinics  are 
scheduled  to  be  installed  and  operational  in  June  2007)  and  radiology  staff  at  all  1 3  sites  were  thoroughly  trained, 

■Measurement  of  efficiency  of  radiologists  and  primary  care  providers  (PCPs)  compared  to  baseline, 

■Measurement  of  satisfaction  of  PCPs  compared  to  baseline, 

■  Measurement  of  efficacy  of  PCPs  providing  the  initial  interpretation  of  radiological  studies  -  prior  to  and  after  the  implementation  of  PACS, 

■Plans  to  install  revised  ARSAM  Systems  Design  Plan  to  address  penetration  issues  identified  during  initial  testing  of  PACS 
■Plans  to  perform  another  penetration  test  to  assess  resistance  to  intrusion  and  device  compromise. 

15.  SUBJbCI  1  fcKMS 

PACS,  ARSAM,  clinical  outcomes 

16.  SECURITY  CLASSIFICATION  OF 


17.  LIMITATION 

18.  NUMBER 

OF  ABSTRACT 

OF  PAGES 

1  HIS  PAGt 

U 

uu 

13 

a.  KbPOKI 

U 


b.  ABSIKACI 

U 


USAMRMC 


19b.  I  ELbPHONb  NUMBER  (include  area 

code) 


Standard  l-orm  29b  (Kev.  8-98) 
Prescribed  by  ANSI  Std.  Z39.18 


Table  of  Contents 


Introduction . 4 

Body . 4 

Key  Research  Accomplishments . 5 

Reportable  Outcomes . 12 

Conclusions . 12 

References . 13 


Appendices 


13 


Introduction 


The  project  period  encompasses  the  third  phase  of  a  three-phase  initiative  to  install  Picture 
Archiving  and  Communication  Systems  (PACS)  and  Teleradiology  at  North  Mississippi  Medical 
Center’s  (NMMC)  main  hospital  campus  in  Tupelo,  four  community  hospitals  in  northeastern 
Mississippi,  and  at  15  clinics  and  to  test  relevant  research  hypotheses  pertaining  to  quality, 
security,  and  clinical  issues. 

Phase  I  involved  1)  installation  of  PACS  equipment  secured  with  Anny  Security  Architecture  for 
Medical  (ARSAM)  Systems  Design  Plan  on  all  freestanding  buildings  on  the  NMMC  campus, 
including  NMMC-Tupelo,  Women’s  Health  Center,  Longtown  Imaging,  Digestive  Health, 

Internal  Medicine  Associates,  and  the  Cancer  Center,  2)  training  of  NMMC’ s  radiologists,  and  3) 
design  of  research  database. 

Phase  II  deliverables  included  1)  installation  of  PACS  equipment  secured  with  ARSAM  Systems 
Design  Plan  at  four  community  hospitals  in  northeastern  Mississippi,  2)  training  of  all  radiology 
staff  members  at  these  facilities,  3)  multiple  tests  of  security  architecture,  and  4)  collection  and 
analysis  of  efficiency,  satisfaction,  and  outcome  data. 

Phase  III  deliverables  include  1)  PACS  equipment  became  fully  operational  at  13  primary  care 
clinics  in  rural  Mississippi  (2  clinics  are  scheduled  to  be  installed  and  operational  in  June  2007), 
2)Radiology  staff  at  all  13  sites  were  thoroughly  trained,  3)  measurement  of  efficiency  of 
radiologists  and  primary  care  providers  (PCPs)  compared  to  baseline,  4)  measurement  of 
satisfaction  of  PCPs  compared  to  baseline,  5)  measurement  of  efficacy  of  PCPs  providing  the 
initial  interpretation  of  radiological  studies  -  prior  to  the  implementation  of  PACS,  6)  plans  to 
install  revised  ARSAM  Systems  Design  Plan  to  address  penetration  issues  identified  during  testing 
of  the  PACS  system  and,  7)  plans  to  perform  another  penetration  test  to  assess  resistance  to 
intrusion  and  device  compromise. 

Body 

Three  years  ago,  NMMC,  in  conjunction  with  its  health  system,  North  Mississippi  Health 
Services  (NMHS),  proposed  a  three-phase  initiative  to  install  Picture  Archiving  and 
Communication  Systems  (PACS)  and  Teleradiology  at  its  main  hospital  campus,  four 
community  hospitals  in  northeastern  Mississippi,  one  community  hospital  in  northwestern 
Alabama,  and  at  15  clinics.  In  addition  to  improving  the  quality  of  radiology  services  for  people 
living  in  this  rural  region,  including  military  personnel,  military  reserve  personnel,  and  military 
dependents,  the  PACS  systems  would  be  used  to  test  security  architecture  for  networked  medical 
devices,  an  initiative  of  critical  importance  to  the  military,  and  to  research  clinical  outcomes 
associated  with  PACS  technology  that  are  of  significance  to  the  military  and  other  health  care 
operations. 

The  hypotheses  to  be  tested  were  that  implementation  of  state-of-the-art  PACS  on  the  NMMC- 
Tupelo  campus,  four  community  hospitals,  and  other  sites  affiliated  with  the  NMHS  system 
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will  1)  improve  radiology  efficiency,  2)  improve  clinical  outcomes,  and  3)  demonstrate  that 

networked  PACS  can  be  made  secure  to  intrusion  and  device  compromise. 

Expected  results  of  the  initiative  included: 

1)  Increased  efficiency  of  radiologists  as  measured  by  perfonnance  indicators 

2)  Decreased  number  of  radiology  support  staff  (film  librarians  and  technicians) 

3)  Increased  satisfaction  of  radiologists,  other  medical  providers,  and  patients  as  measured 
by  satisfaction  surveys  (medical  staff,  radiologists,  and  consumers) 

4)  Identification  of  highly  effective  security  architecture  for  networked  PACS  systems  that 
can  be  replicated  for  Army  Medicine’s  networked  medical  devices,  including  PACS 

5)  Improved  clinical  outcomes  as  measured  by  specific  radiology  indicators  on  the  length  of 
stay  (LOS)  and  other  indicators  for  CAP  and  stroke 

Potential  benefits  of  the  three-phase  project  included: 

•  Demonstration  of  radiology  service  management  at  remote  locations 

•  Information  on  the  impact  of  PACS  technology  on  clinical  outcomes  for  CAP  and  stroke 

•  Demonstration  of  secure  teleradiology  services  at  remote  locations 

The  project’s  relevance  to  the  military  is  related  to  its  capacity  to: 

•  Simulate  the  military’s  management  of  radiology  services  in  remote,  noncombat  settings 

•  Validate  ARSAM  security  architecture  for  PACS 

•  Ensure  availability  of  high  quality  radiological  services  for  military  personnel,  reserve 
personnel,  and  dependents 

Project  Period  Deliverables  and  Results 

Most  deliverables  for  this  project  period  were  met  during  the  period  from  April  28,  2006  to  April 

28,  2007.  As  noted  below,  2  clinic  installations  are  not  yet  complete  and  until  installation  is 

complete  penetration  testing  cannot  be  perfonned. 

•  PACS  equipment  became  fully  operational  at  13  rural  primary  care  clinics:  Pulmonary 
Clinic,  Family  Medicine  Residency  Center,  Bames  Crossing,  Med  Serve,  Eupora  FMC, 
Okolona  FMC,  North  Columbus  FMC,  Lowndes  FMC,  Pontotoc  FMC,  Fulton  FMC, 
Baldwyn  FMC,  Oxford  FMC,  Chickasaw  FMC  (Hamilton  Medical  Clinic  and  Saltillo  FMC 
are  being  implemented  in  June  2007) 

•  Radiology  staff  at  all  13  sites  were  thoroughly  trained. 

•  Measurement  of  efficiency  of  radiologists  and  primary  care  providers  (PCPs)  compared  to 
baseline, 

•  Measurement  of  satisfaction  of  PCPs  compared  to  baseline, 

•  Measurement  of  efficacy  of  PCPs  providing  the  initial  interpretation  of  radiological  studies 
-  prior  to  and  after  the  implementation  of  PACS, 
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•  Once  the  clinic  installation  is  complete,  the  ARSAM  System  Design  Plan  network  security 
changes  will  be  installed, 

•  Network  vulnerability  assessment  and  penetration  tests  are  being  scheduled  for  August  or 
September  2007  to  assess  resistance  to  intrusion  and  device  compromise. 


Key  Research  Accomplishments 

>  Research  Area:  Impact  of  PACS  on  efficiency  of  radiologists  as  measured  by 
performance  indicators 

Dramatic  reductions  were  noted  in  Report  Turnaround  Time,  the  time  that  elapses  following 
completion  of  the  procedure  and  receipt  of  the  radiologist’s  interpretation  of  the  image,  pre-  and 
post-installation  of  PACS  capabilities.  We  randomly  selected  60  patients  who  received  a  chest 
x-ray  at  one  of  the  clinics  prior  to  and  after  the  implementation  of  PACS.  The  pre -PACS 
timeframe  was  January  2006  and  the  post-PACS  timeframe  was  December  2006.  Prior  to  the 
implementation  of  PACs  in  the  clinics,  an  x-ray  would  be  performed  in  the  clinics  and  then 
transported  by  courier  to  the  radiologist  in  Tupelo  (a  distance  of  over  90  miles  for  the  most 
remote  clinic).  The  Tupelo-based  radiologist  would  interpret  the  study  and  send  the 
interpretation  via  the  electronic  medical  record  (EMR)  that  links  the  hospitals  and  clinics.  A 
chart  review  of  the  EMR  was  perfonned  and  the  following  elements  were  collected  to  examine 
the  efficiency  of  the  PACS  system  in  the  clinics  (see  figure  1  for  results): 

>  When  the  patient  visit  occurred  (date  and  time  -  in  minutes ); 

>  When  the  radiologist’s  interpretation  was  available  in  the  EMR; 

>  When  the  PCP’s  signature  appeared  on  the  radiologist’s  interpretation  (to  indicate 
the  PCP  had  seen  the  radiologist ’s  interpretation) ;  and 

>  When  the  final  primary  care  provider  (PCP)  signature  appeared  on  the  overall 
visit  (to  indicate  the  PCP ’s  closure  on  this  episode). 

A  2-sample  t-test,  adjusted  for  unequal  variance,  where  appropriate,  was  used  to  calculate  the  p- 
values: 


Figure  1:  Pre  and  Post-PACS  efficiency  in  NMHS  clinic  setting 


Pre-PACS  (hrs) 

Post-PACS(hrs) 

p-value 

Radiologist’s  interpretation  available  on 
EMR 

49 

1.6 

<.0001 

PCP  signs  off  on  radiologist’s 
interpretation  in  EMR 

44 

35 

.2406 
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PCP  final  sign  off  on  visit 

90 

37 

<.0001 

It  appears  that  the  radiologists  have  significantly  reduced  the  turnaround  time  for  reporting  their 
interpretations  on  patients’  EMR’s.  The  PCPs,  however,  have  not  significantly  reduced  the  time 
before  signing  off  on  having  read  the  radiologists’  interpretations.  The  PCP’s  final  sign  off  on 
the  patient’s  chart,  however,  did  reduce  significantly. 

>  Research  Area:  Impact  of  PACS  on  costs  for  film  reduction  and  decreased  courier 
usage. 

The  North  Mississippi  Medical  Clinics  Inc.  (NMMCI)  administrators  perfonned  this  evaluation 
They  noted  a  decrease  in  film  costs,  but  an  overall  increase  in  their  MIS  support  fees  and 
increased  bandwith  costs.  These  increases  more  than  outweigh  the  film  reduction  savings  and 
since  January  2007,  their  radiology  costs  are  up  by  22%  since  the  implementation  of  PACS. 
There  is  no  decrease  in  courier  costs,  since  the  courier  is  still  transporting  other  materials 
between  NMMC-Tupelo  and  the  clinics. 

>  Research  Area:  Impact  of  PACS  on  satisfaction  of  primary  care  provides  (clinic- 
based  physicians  and  nurse  practitioners). 

In  June  2007,  25  PCP’s  in  the  13  clinics  that  have  implemented  PACS  returned  the  survey. 

This  is  a  5-point  survey  with  the  PCP  rating  5  for  “excellent”  and  1  for  “poor”.  Their  results  are 
compiled  and  compared  to  the  baseline  survey  that  was  completed  with  PCP’s  from  the  entire 
region.  Note,  the  original  pre-PACS  questions  primarily  referred  to  radiologic  studies  being 
performed  at  the  hospitals.  In  each  of  the  four  categories  of  satisfaction  questions,  the 
cumulative  score  increased  from  the  baseline  survey  in  2003  to  post-PACS  in  2007.  Refer  to 
Figure  2. 


Figure  2:Clinic-based  PCP  Satisfaction  with  PACS  Survey  Results 


Referring  Pliysicians/CIinic-based  PCPs 

2003 

2007 

Scheduling  &  Registration 

3.65 

4.42 

Time  it  takes  to  schedule  outpatients 

4.40 

Responsiveness  for  urgent  patients 

4.44 

Physician  Needs 

3.74 

4.66 

Availability  of  radiologists  to  review  films 

4.76 

Timeliness  of  receiving  preliminary  reports 

4.60 

Timeliness  of  receiving  final  reports 

4.72 

Timeliness  in  completion  of  urgent  exams 

4.76 

Report  turn-around  time  for  urgent  patients 

4.64 
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Referring  Physicians/Clinic-based  PCPs 

2003 

2007 

Patient  Needs 

3.72 

4.43 

Waiting  time  in  the  department 

4.43 

Customer  service  attitude  &  tech  perfonnance 

4.430xytoc 
in,  Basic 
Behavioral 
Research, 
Relationshi 
p,  Sexual 
Activity, 
Longitudin 
al,  Human 
Subjects 

Radiologists 

3.93 

4.48 

Quality  &  accuracy  of  interpretation 

4.74 

Availability  for  consultation 

4.64 

Calling  of  positive  reports  to  physicians 

4.08 

To  supplement  the  empirical  data,  which  trends  strongly  to  greater  satisfaction  with  radiology 
services,  PCP  respondents  write-in  comments  are  presented  in  Figure  3: 

Figure  3:  Clinic-based  PCP  Survey  Write-in  Comments  (2007) 


If  you  have  checked  off  any  “needs  improvement”  or  “poor,”  please  tell  us  why 

•  None  noted 


Do  you  feel  that  the  implementation  of  PACS  has  changed  the  way  you  treat 

patients?  For  example,  if  the  turn  around  time  for  reports  is  faster,  has  this  helped 

with  your  ability  to  diagnose  and  treat  patients? 

•  Yes,  excellent  results  and  service.  Able  to  make  diagnosis  quicker. 

•  Much  improved,  positive  improvement. 

•  It  is  very  helpful  when  the  report  comes  back  while  the  patient  is  still  here  (this 
happens  sometimes,  but  not  always).  Overall,  I  am  very  happy  with  the  system  and 
like  it  much  better  than  hard  copy  films.  Especially  nice  to  be  able  to  discuss  with 
radiologist  on  phone  while  looking  at  film. 

•  It  has  been  very  helpful  sometimes  to  have  radiologist  report  in  hand  when  discussing 
former  radiographic  abnormalities  with  patients. 

•  Not  really. _ 


Please  provide  us  with  any  comments  or  suggestions  you  have  regarding  imaging 
services. 


•  At  first  I  wasn’t  sure  how  this  was  going  to  be  helpful.  Now  after  using  PACS  and 
seeing  how  quick  it  is  and  how  fast  we  get  the  reports  back,  I  love  it! 

•  Keep  up  the  good  turn  around  time  on  film  review  by  physician. 

•  I  have  enjoyed  the  ability  to  magnify  images;  service  has  been  faster  for  patients. 

•  Great  service. 

•  Please  continue. 


Research  Area:  Improved  clinical  outcomes 

During  the  randomly-selected  60  patient  pre-  and  post-PACS  chart  review,  two  more  data 
elements  were  collected  to  examine  the  clinical  efficacy  of  the  PACS  system  in  the  clinics: 

•  If  the  PCP  documented  his/her  own  interpretation  of  the  study;  and 

•  If  the  PCP’s  and  the  radiologist’s  interpretation  agreed  with  each  other. 

A  chi  square  test  was  performed  to  determine  the  p-value. 


Figure  4:  Clinical  impact  of 

PACS  of  NMHS  clinic  patients 

Pre-PACS  (%) 

Post-PACS  (%) 

p-value 

Cases  of  PCPs  who  documented  their 
interpretation  of  their  patients’  chest  x-ray 

28  pts  (47%) 

28  pts  (47%) 

1.00 

Cases  of  agreement  between  PCPs 
interpretation  and  the  radiologists’ 
interpretation 

25  pts  (89%) 

24  pts  (86%) 

1.00 

One  of  the  basic  premises  of  implementing  PACS  in  the  clinics  is  to  provide  the  clinic-based 
PCP’s  with  the  radiologist’s  interpretation  during  the  patient’s  visit.  As  noted  in  Figure  1,  the 
turnaround  time  of  the  arrival  of  the  radiologist’s  interpretation  on  the  patient’s  EMR  decreased 
from  49.0  to  1.6  hours.  This  incredible  improvement,  however,  did  not  change  the  frequency  of 
the  PCP’s  recording  their  own  interpretations  on  the  EMR  (47%  pre  and  post  PACS).  And, 
understandably,  it  did  not  appreciably  change  the  agreement  between  the  PCP  and  radiologists 
(89%  pre  and  86%  post  PACS). 

It  is  not  possible  to  assess  the  actual  impact  on  the  PCP’s  use  of  the  information  they  received. 
Based  on  their  comments  in  the  satisfaction  survey  (Figure  3),  it  appears  that  the  physicians  are 
using  the  radiologists’  interpretations  to  make  their  treatment  decisions. 

Each  of  the  seven  cases  of  disagreement  between  the  PCP’s  and  the  radiologist’s  interpretation 
of  the  patients’  chest  x-ray  (CXR)  was  reviewed  by  the  clinic  system’s  quality  improvement 
physician.  He  determined  whether  there  was  any  clinical  impact  on  the  patient’s  care  because  of 
the  disagreement. 
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Pre- 

PACS 

Cases 

Case  Description 

Clinical 

Impact 

1 

The  PCP  interpreted  the  CXR  of  52  yo  male  with  coughing 
and  wheezing  as  a  “little  patch  of  pneumonitis”  and  prescribed 
antibiotics  for  bronchitis.  The  radiologist  interpreted  the  CXR 
as  “within  normal  limits.” 

None  -  would 
also  treat  these 
clinical  symptoms 
with  antibiotics 

2 

The  PCP  interpreted  the  CXR  of  16  yo  female  with  back  pain 
in  her  left  flank  area  as  “possible  costochrondral  tear”  and 
treated  her  with  a  mild  pain  medicine  and  an  anti¬ 
inflammatory  agent.  The  radiologist  interpreted  the  CXR  as  a 
“negative  examination.” 

None-  would  also 
treat  the  pt’s  pain 
with  pain  med  & 
anti-inflammatory 
agents 

3 

The  PCP  interpreted  the  CXR  of  68  yo  female  with  cough, 
cold  and  congestion  “infiltrate  of  left  lower  lung”  and 
prescribed  antibiotics  for  bronchitis.  The  radiologist 
interpreted  the  CXR  as  “within  nonnal  limits.” 

None  -  would 
also  treat  these 
clinical  symptoms 
with  antibiotics 

Post- 

PACS 

Cases 

Case  Description 

Clinical 

Impact 

1 

The  PCP  interpreted  the  CXR  of  57  yo  male  with  cough,  chest 
congestion  and  wheezing  as  “chronic  changes  and  light 
bronchial  cuffing”  and  prescribed  antibiotics  and  an  inhaler  for 
bronchitis.  The  radiologist  interpreted  the  CXR  as 
“unremarkable  chest.” 

None  -  would 
also  treat  these 
clinical  symptoms 
with  antibiotics 
and  an  inhaler 

2 

The  PCP  interpreted  the  CXR  of  6  yo  female  who  is  on  a 
return  visit  for  her  bad  cough  as  “a  suspicious  area  in  the  left 
base  for  an  infiltrate”  and  prescribed  a  different  antibiotic  and 
a  corticosteroid  for  possible  pneumonia.  The  radiologist 
interpreted  the  CXR  as  “and  infiltrate  in  the  lingular  segment 
of  the  left  upper  lobe  . . .  felt  to  represent  a  left  upper  lobe 
pneumonis.” 

None-  it  is 
difficult  to  make 
fine  anatomical 
distinctions  in  a 
small  child.  The 
interpretations 
were  similar  and 
the  treatment 
would  be  the 

same. 

3 

The  PCP  interpreted  the  CXR  of  70  yo  female  with  lung 
congestion  as  “no  consolidations,  infiltrates  or  effusions”  and 
prescribed  antibiotics  and  cough  medicine  for  bronchitis.  The 
radiologist  interpreted  the  CXR  as  “a  slight  increase  in  the  size 
of  the  vague  nodular  opacity  in  the  right  upper  lobe.” 

Possible  -  would 
require  a  CT  of 
the  chest  to 
examine  nodular 
opacity. 

Follow-up :  PCP 
ordered  the  chest 
CT  the  following 
day. 

4 

The  PCP  interpreted  the  CXR  of  9  yo  female  with  nighttime 
cough  and  shortness  of  breath  as  “clear”  and  prescribed 

None  -  different 
terminology  but 
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antibiotics,  cough  medicine  and  a  corticosteroid  for  bronchitis,  same  diagnosis 
The  radiologist  interpreted  the  CXR  as  “peribronchial  wall  and  treatment. 
change  consistent  with  bronchitis.” _ 


From  this  relatively  small  sample,  it  does  not  appear  that  PACS  has  made  any  difference  in  the 
clinic  patients’  clinical  outcomes. 

While  PACS  increased  the  speed  with  which  the  radiologists’  returned  their  interpretations,  it 
did  not  demonstrate  an  impact  on  clinical  outcomes  in  the  current  study.  In  retrospect,  we  may 
have  assessed  the  impact  too  soon  after  implementation  to  see  any  difference  in  PCP  behavior. 
Although  PCP’s  indicated  they  are  looking  at  the  radiologists’  studies  before  making  their 
decisions  (per  satisfaction  comments),  they  did  not  appear  to  sign  off  on  the  radiologists’  notes 
significantly  earlier  than  before.  This  may  indicate  that  many  of  the  PCP’s  have  not  yet 
incorporated  this  new  technology  into  their  practices.  We  may  want  to  reassess  the  clinics  after 
they  have  had  more  than  1-2  months  of  experience  using  PACS. 

Project  Period  Security  Architecture  Testing 

>  Research  Area:  Identification  of  highly  effective  security  architecture  for 

networked  PACS  systems  that  can  be  replicated  for  Army  Medicine’s  networked 
medical  devices,  including  PACS 

BACKGROUND  regarding  phase-2  evaluation:  NMMC  contracted  Sword  &  Shield  Enterprise 
Security,  Inc.  (Sword  &  Shield)  to  conduct  a  network  vulnerability  assessment  and  penetration 
test  of  NMMC’ s  external  and  internal  networks.  The  assessment  examined  NMMC 's 
implementation  of  the  Army  Security  Architecture  for  Medical  (ARSAM),  a  defense-in-depth 
network  security  architecture  for  FDA-approved  medical  devices.  NMMC  is  using  ARSAM  in  its 
deployment  of  the  PACS  within  its  hospital  information  system  network. 

Sword  and  Shield  Findings  from  May  2006: 

•  From  an  external  perspective,  the  overall  security  posture  of  the  NMMC  network  was 
deemed  to  be  satisfactory.  The  firewall  and  network  devices  provided  an  adequate  level 
of  protection  to  the  NMMC  internal  network,  and  PACS  network,  from  Internet-based 
threats. 

•  However,  Sword  and  Shield  was  able  to  circumvent  the  ARSAM  protections  implemented 
at  NMMC  during  the  internal  phase  of  testing  by  taking  advantage  of  some  PACS  devices 
that  were  located  outside  of  the  protected  enclave  and  also  by  taking  advantage  of  other 
devices  on  the  network  which  were  not  patched  with  all  of  the  latest  security  patches  or 
system/applications  updates. 

CURRENT  STATUS:  The  recommended  security  patches  have  been  installed  into  the  PACS 
system.  Once  the  PACS  systems  are  installed  at  the  final  two  clinics,  NMHS  will  contact  Sword 
and  Shield  to  request  that  they  conduct  a  second  network  vulnerability  assessment  and 
penetration  test  of  NMMC’ s  external  and  internal  networks. 

Reportable  Outcomes 
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To  date,  no  manuscripts,  abstracts,  grant  applications,  patents,  etc.  have  been  submitted  that  are 
direct  results  of  this  project. 


Conclusions 

Implementation  of  PACS  technology  on  NMMC’s  campus  in  Tupelo,  four  community  hospitals 
and  13  clinics  has  yielded  information  of  potential  value  to  other  health  care  agencies  serving 
rural  regions: 

Current  (Phase-3)  Report 

1.  PACS  technology  is  associated  with  dramatic  decreases  in  radiological  report  turn¬ 
around  in  rural  clinic  settings. 

2.  Clinic-based  PCP’s  reported  greater  satisfaction  with  radiological  services  following 
introduction  of  PACS  capabilities. 

3.  The  implementation  of  PACS  did  not  affect  the  frequency  of  PCP’s  interpreting  their 
patients’  CXR’s. 

4.  The  cost  of  PACS  appears  to  be  higher  for  the  clinic  system. 

5.  The  implementation  of  PACS  did  not  affect  the  interpretation  of  the  CRX’s  or  the  clinical 
treatment  the  patients  received. 

Previous  (Phase-2)  Report 

1.  PACS  technology  is  associated  with  dramatic  decreases  in  radiological  report  turn¬ 
around  in  rural  hospital  settings. 

2.  At  two  of  the  four  community  hospital  sites  (50  percent),  the  introduction  of  PACS 
technology  resulted  in  cost  savings  (personnel  and  film) 

3.  Referring  physicians  reported  greater  satisfaction  with  radiological  sendees  following 
introduction  of  PACS  capabilities. 

4.  Data  analyzed  to  date  does  not  support  the  hypothesis  that  the  use  of  PACS  technology 
increases  the  speed  with  which  antibiotics  are  administered  as  indicated  for  CAP. 

5.  The  low  usage  of  tPA  in  the  management  of  ischemic  stroke  makes  it  an  unreliable 
process  indicator  for  efficacy  of  PACS  in  the  management  of  stroke. 

6.  Overall  outcomes  (length  of  stay,  patient  mortality  and  median  cost)  trended  towards 
improvement  in  the  post-PACS  period  for  both  CAP  and  stroke. 

7.  Network  sensitivity  and  penetration  testing  revealed  that  ARSAM protections 
implemented  at  NMMC  were  inadequate  and  that  additional  precautions  needed  to  be 
taken  in  the  security  system  architecture. 

As  required  by  the  Research  Technical  Reporting  Requirements,  “So  what?”  Work  completed 
during  the  three  phases  of  PACS  implementation  and  research  at  NMMC  does  indeed  have 
implications  for  rural  hospitals/health  systems  and  for  the  military.  PACS  technology  has 
dramatically  increased  the  tum-around  time  for  radiological  images  and  increased  significantly  the 
number  of  “first  reads”  made  by  radiologists  rather  than  other  physician  specialists.  PACS 
technology  will  likely  be  instrumental  in  improving  physician  satisfaction  with  radiological  services 
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at  other  rural  clinics  and  hospitals,  which  struggle  to  recruit  and  retain  qualified  health  care 
providers.  When  contemplating  implementation  of  PACS,  some  hospitals  may  be  able  to  project 
cost  savings  from  personnel  or  supply  (film)  budgets,  however,  our  Phase-3  clinic-based  study 
actually  showed  an  increase  in  costs  due  to  additional  IS  support  costs  over  film  savings. 

The  results  of  network  sensitivity  and  penetration  testing  completed  to  date  at  NMMC  suggest  that 
ARSAM  protections  must  be  enhanced  to  guard  patient  data,  which  will  guide  other  rural  hospitals 
in  the  implementation  and  design  of  their  PACS  systems.  Test  results  are  also  of  significance  to  the 
military’s  efforts  to  deliver  radiological  services  from  remote,  noncombat  sites  to  battlespaces. 

The  health  system’s  experiences  will  be  used  to  formulate  recommendations  for  installation, 
implementation,  and  application  of  PACS  systems,  including  highly  effective  security 
architecture.  Data  collected  on  indicators  pertinent  to  clinical  outcomes  will  be  used  to 
document  the  impact  of  PACS  technology  in  civilian  applications. 
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